Audiovisual spatial recalibration but not integration is shaped by early sensory experience.

To clarify the role of sensory experience during early development for adult multisensory learning capabilities, we probed audiovisual spatial processing in human individuals who had been born blind because of dense congenital cataracts (CCs) and who subsequently had received cataract removal surgery, some not before adolescence or adulthood. Their ability to integrate audio-visual input and to recalibrate multisensory spatial representations was compared to normally sighted control participants and individuals with a history of developmental (later onset) cataracts. Results in CC individuals revealed both normal multisensory integration in audiovisual trials (ventriloquism effect) and normal recalibration of unimodal auditory localization following audiovisual discrepant exposure (ventriloquism aftereffect) as observed in the control groups. In addition, only the CC group recalibrated unimodal visual localization after audiovisual exposure. Thus, in parallel to typical multisensory integration and learning, atypical crossmodal mechanisms coexisted in CC individuals, suggesting that multisensory recalibration capabilities are defined during a sensitive period in development.

Crossmodal associations modulate multisensory spatial integration.

According to the Bayesian framework of multisensory integration, audiovisual stimuli associated with a stronger prior belief that they share a common cause (i.e., causal prior) are predicted to result in a greater degree of perceptual binding and therefore greater audiovisual integration. In the present psychophysical study, we systematically manipulated the causal prior while keeping sensory evidence constant. We paired auditory and visual stimuli during an association phase to be spatiotemporally either congruent or incongruent, with the goal of driving the causal prior in opposite directions for different audiovisual pairs. Following this association phase, every pairwise combination of the auditory and visual stimuli was tested in a typical ventriloquism-effect (VE) paradigm. The size of the VE (i.e., the shift of auditory localization towards the spatially discrepant visual stimulus) indicated the degree of multisensory integration. Results showed that exposure to an audiovisual pairing as spatiotemporally congruent compared to incongruent resulted in a larger subsequent VE (Experiment 1). This effect was further confirmed in a second VE paradigm, where the congruent and the incongruent visual stimuli flanked the auditory stimulus, and a VE in the direction of the congruent visual stimulus was shown (Experiment 2). Since the unisensory reliabilities for the auditory or visual components did not change after the association phase, the observed effects are likely due to changes in multisensory binding by association learning. As suggested by Bayesian theories of multisensory processing, our findings support the existence of crossmodal causal priors that are flexibly shaped by experience in a changing world.

Multisensory Integration Develops Prior to Crossmodal Recalibration.

It has been hypothesized that crossmodal recalibration plays a crucial role for the development of multisensory integration capabilities [1]. To test the developmental trajectory of multisensory integration and crossmodal recalibration, we used a combined ventriloquist/ventriloquist aftereffect paradigm [2] in children aged 5-9 years. The ventriloquist effect (indicating multisensory integration), that is, the shift of auditory localization toward simultaneously presented but spatially discrepant visual stimuli, was larger in children than in adults, which was attributed to a lower auditory localization precision in the children. In fact, the size of the ventriloquist effect depended on the visual stimulus reliability in both children and adults. In all groups, the ventriloquist effect was best explained by a causal inference model. In contrast to their multisensory integration capabilities, 5-year-old children did not recalibrate. The immediate ventriloquist aftereffect (indicating recalibration after a single exposure to a spatially discrepant audio-visual stimulus) emerged in 6- to 7-year-old children, whereas the cumulative ventriloquist aftereffect (reflecting recalibration to the audio-visual spatial discrepancies over the complete experiment) was not observed before the age of 8 years. First, in contrast to common beliefs, the present results provide evidence that multisensory integration precedes rather than follows crossmodal recalibration during development. Second, we report developmental evidence for a dissociation of the processes involved in multisensory integration and immediate as well as cumulative recalibration. We speculate that multisensory integration is a prerequisite for crossmodal recalibration, because the multisensory percept, rather than unimodal cues, might comprise a crucial signal for the calibration of the sensory systems.

A Survey on Probabilistic Models in Human Perception and Machines.

Extracting information from noisy signals is of fundamental importance for both biological and artificial perceptual systems. To provide tractable solutions to this challenge, the fields of human perception and machine signal processing (SP) have developed powerful computational models, including Bayesian probabilistic models. However, little true integration between these fields exists in their applications of the probabilistic models for solving analogous problems, such as noise reduction, signal enhancement, and source separation. In this mini review, we briefly introduce and compare selective applications of probabilistic models in machine SP and human psychophysics. We focus on audio and audio-visual processing, using examples of speech enhancement, automatic speech recognition, audio-visual cue integration, source separation, and causal inference to illustrate the basic principles of the probabilistic approach. Our goal is to identify commonalities between probabilistic models addressing brain processes and those aiming at building intelligent machines. These commonalities could constitute the closest points for interdisciplinary convergence.

An Adaptation-Induced Repulsion Illusion in Tactile Spatial Perception.

Following focal sensory adaptation, the perceived separation between visual stimuli that straddle the adapted region is often exaggerated. For instance, in the tilt aftereffect illusion, adaptation to tilted lines causes subsequently viewed lines with nearby orientations to be perceptually repelled from the adapted orientation. Repulsion illusions in the nonvisual senses have been less studied. Here, we investigated whether adaptation induces a repulsion illusion in tactile spatial perception. In a two-interval forced-choice task, participants compared the perceived separation between two point-stimuli applied on the forearms successively. Separation distance was constant on one arm (the reference) and varied on the other arm (the comparison). In Experiment 1, we took three consecutive baseline measurements, verifying that in the absence of manipulation, participants' distance perception was unbiased across arms and stable across experimental blocks. In Experiment 2, we vibrated a region of skin on the reference arm, verifying that this focally reduced tactile sensitivity, as indicated by elevated monofilament detection thresholds. In Experiment 3, we applied vibration between the two reference points in our distance perception protocol and discovered that this caused an illusory increase in the separation between the points. We conclude that focal adaptation induces a repulsion aftereffect illusion in tactile spatial perception. The illusion provides clues as to how the tactile system represents spatial information. The analogous repulsion aftereffects caused by adaptation in different stimulus domains and sensory systems may point to fundamentally similar strategies for dynamic sensory coding.